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Abstract 

Mobile  ad-hoc  networks  offer  dynamic  portable  communication  with  little  or  no 
infrastructure.  While  this  has  many  benefits,  there  are  additional  shortcomings  specific  to 
wireless  communication  that  must  be  addressed.  This  research  proposes  gossip-based  on- 
demand  key  distribution  as  a  means  to  provide  data  encryption  for  mobile  ad-hoc 
networks. 

This  technique  uses  message  keys  to  avoid  encrypting  and  decrypting  a  message 
at  every  node.  Other  optimizations  used  include  secure  channel  caching  and  joint  rekey 
messages.  The  use  of  gossip  makes  the  scheme  robust  to  node  failure. 

Experimental  results  show  only  a  15%  increase  in  end-to-end  delay  with  a  node 
failure  rate  of  10%.  The  percentage  of  messages  successfully  delivered  to  nodes  stays 
between  91-98%  under  the  same  10%  node  failure  rate.  The  network  load  is  distributed 
to  all  nodes  in  the  group  preventing  overload  and  single  points  of  failure. 
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ON-DEMAND  KEY  DISTRIBUTION  FOR  MOBILE  AD-HOC  NETWORKS 


I.  Introduction 


1.1  Background 

Mobile  ad-hoc  networks  (MANETs)  have  the  potential  to  provide  communication 
to  a  large  group  of  nodes  while  decreasing  the  amount  of  infrastructure  needed  to 
establish  and  maintain  communication.  Certain  applications  have  communication 
requirements  that  can  only  be  satisfied  by  wireless  networking.  Wireless  networking  is 
used  everyday  when  talking  on  a  cell  phone  or  surfing  the  Internet  on  a  laptop  at  a  coffee 
shop.  These  common  applications  often  have  some  sort  of  infrastructure.  This  may  be  a 
wireless  access  point  in  the  coffee  shop  or  the  thousands  of  cell  phone  towers  on  top  of 
buildings,  beside  highways,  or  upon  hills. 

However,  in  many  cases,  especially  in  military  applications,  it  is  not  feasible  to 
rely  so  heavily  on  an  established  network  infrastructure.  When  entering  hostile  territory, 
the  infrastructure  may  not  exist  or  the  existing  infrastructure  may  not  be  available. 
Furthermore,  establishing  an  infrastructure  can  be  costly  and  will  likely  become  a  target 
for  the  enemy.  MANETs  enable  communication  wherever  nodes  are  present  avoiding  the 
costs  and  limitations  of  a  fixed  infrastructure. 

Much  of  the  current  research  and  networking  protocols  assume  a  wired  network. 
Moving  from  a  wired  to  a  wireless  environment  introduces  considerations  that  are  not  a 
concern  in  wired  networks.  One  such  consideration  is  the  likelihood  nodes  will 
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experience  intermittent  connections.  The  connection  problem  could  be  isolated  to  a 
single  link,  like  when  an  obstacle  blocks  transmission.  The  problem  could  also  affect  all 
links  for  a  particular  area,  perhaps  due  to  interference.  Reliable  group  communication 
protocols  attempt  to  send  all  updates  to  intermittent  nodes.  Sending  to  a  node  during  its 
down  time  can  increase  the  load  on  the  network. 

Gossip-based  protocols  communicate  using  epidemic  communication  patterns. 
Messages  are  distributed  to  randomly  selected  nodes  and  probabilistically  delivered  to  all 
nodes  depending  on  each  nodes  connectedness  to  the  network,  known  as  their  health.  A 
healthy  node  will  likely  receive  the  desired  amount  of  updates,  while  an  unhealthy  node 
will  likely  receive  only  as  many  updates  as  its  intennittent  links  will  allow.  Gossip-based 
networking  attempts  to  overcome  the  unreliability  of  wireless  networks. 

Many  applications  require  secure  networks  in  which  nodes  transmit  data  in 
encrypted  form.  This  is  particularly  essential  for  wireless  communication  as  all 
transmissions  are  subject  to  monitoring.  Implementing  security  is  challenging  when 
using  group  communication,  and  is  further  complicated  when  operating  over  wireless 
networks.  A  key  distribution  scheme  for  such  environments  must  ensure  all  nodes  to 
receive  keys  in  a  timely,  efficient  manner.  The  scheme  must  also  be  able  to  support  any 
unhealthy  nodes  in  the  network. 

1.2  Research  Objectives 

This  research  develops  a  key  distribution  scheme  that  adds  data  encryption  to  a 
gossip-based  protocol.  The  scheme  adds  little  overhead  to  the  network  and  delivers  keys 
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to  the  extent  that  the  health  of  nodes  allows.  This  thesis  demonstrates  a  security  protocol 
and  compares  it  to  a  secure  reliable  Iolus-based  distribution  scheme. 

1.3  Overview 

Chapter  2  provides  background  information  on  network  encryption  protocols  and 
group  communication.  Chapter  3  states  the  problem,  introduces  a  method  for  overlaying 
security  on  a  gossip-based  protocol,  and  discusses  the  experiments  to  analyze  this 
approach.  Chapter  4  presents  the  data  resulting  from  the  experiments.  Finally,  Chapter  5 
presents  conclusions  and  proposes  areas  of  further  research  to  improve  the  protocol. 
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II.  Literature  Review 


2.1  Chapter  Overview 

This  chapter  reviews  topics  in  gossip-based  key  distribution  protocol  including 
security  for  unicast  communication  and  how  it  relates  to  group  communication. 

Multicast  communication  is  also  discussed  and  current  research  on  secure  group  network 
protocols  considered. 

2.2  Background 
2.2.1  Unicast  Security 

Unicast  communication  is  network  communication  in  its  most  fundamental  fonn, 
one  node  communicating  with  another  node.  Transmitted  data  is  considered  secure  if  no 
node  other  than  the  receiver  is  able  to  interpret  the  data.  Securing  unicast  communication 
can  be  accomplished  in  various  ways.  For  instance,  a  dedicated  line  can  be  used  for 
secure  transmission  assuming  the  line  is  not  compromised  through  some  form  of 
wiretapping.  While  this  has  inherent  benefits,  the  use  of  a  dedicated  line  is  rarely  feasible 
as  a  cost-effective  solution.  More  importantly,  dedicated  lines  are  impossible  if  nodes  are 
mobile.  Therefore,  nearly  all  network  communication  is  done  over  shared  access  media. 
Whether  the  communication  is  wired  or  wireless,  other  nodes  will  receive  the  data  being 
transmitted. 

Several  security  protocols  allow  nodes  to  use  a  shared  network  securely.  Nearly 
all  network  security  protocols  fall  into  to  two  major  categories,  public  key  or  symmetric 
key.  Public  key  algorithms  use  separate  keys  for  encryption  and  decryption.  Symmetric 
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key  algorithms  use  a  single  key  for  encryption  and  decryption  and  are  typically  much 
faster  than  public  key  algorithms. 

2.2. 1.1  Public  Key 

The  Rivest,  Shamir,  and  Adleman  (RSA)  public  key  encryption  algorithm 
[Rivest78]  exploits  the  complexity  of  factoring  a  product  of  large  primes  to  make 
deciphering  encrypted  data  very  difficult.  Two  keys  are  used  in  the  transmission  of  data. 
One  key  is  made  public  and  is  used  by  other  nodes  to  encrypt  data  that  will  be  sent  to  the 
node  to  which  the  private  key  belongs.  The  other  key  is  kept  private  and  is  used  solely 
by  the  receiving  node.  The  node  decrypts  the  encrypted  data  sent  to  it  using  the  private 
key.  Due  to  the  complexity  of  the  math  involved  in  the  encryption  and  decryption 
processes,  the  RSA  algorithm,  like  many  public  key  encryption  algorithms,  tends  to  be 
slow  compared  to  symmetric  key  algorithms. 

Using  public  key  algorithms,  nodes  can  send  encrypted  data  to  a  receiving  node  as 
well.  Each  sending  node  uses  the  same  public  key  to  encrypt  the  data  being  sent  to  a 
single  receiving  node.  Confidentiality  is  maintained  since  the  only  node  capable  of 
decrypting  the  data  is  the  node  that  has  the  private  key. 

2.2. 1.2  Symmetric  Key 

Symmetric  key  algorithms,  such  as  Rivest  Cipher  5  (RC5)  [Rivest95],  are  often 
much  less  computationally  intensive  than  public  key  encryption  algorithms  making 
symmetric  key  algorithms  faster  and  easier  to  implement  than  public  key  algorithms.  The 
principle  shortcoming  is  the  algorithms  require  a  secret  key  be  shared  by  the  nodes  prior 
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to  encrypting  data.  Therefore,  a  secret  channel  between  the  nodes  must  be  used  to 
exchange  the  shared  key. 

Sharing  a  symmetric  key  carries  a  much  greater  significance  than  sharing  a  public 
key.  A  node  receiving  a  symmetric  key,  in  addition  to  encrypting  data,  will  be  able  to 
decrypt  everything  encrypted  with  that  symmetric  key.  Once  a  key  has  been  shared  with 
a  node,  that  node  has  access  to  all  data  encrypted  with  that  key;  future,  present,  and  past. 
For  two  nodes,  this  means  very  little.  A  key  is  generated  and  shared  at  the  beginning  of 
the  session.  When  the  session  is  done,  the  sender  drops  the  shared  key.  Confidentiality 
problems  occur  if  a  symmetric  key  is  reused  in  sessions  with  other  nodes.  Further 
considerations  must  be  made  when  three  or  more  nodes  are  communicating  using  a  single 
symmetric  key  as  will  be  discussed  later. 

2.2.2  Group  Communication 

Group  communication  is  network  communication  between  a  specific  set  of  nodes. 
There  is  a  growing  need  to  distribute  data  to  large  groups  of  nodes.  From  streaming 
media  to  reconnaissance  video,  there  are  applications  in  both  the  commercial  and  defense 
arenas.  Group  communication  in  a  unicast  manner  is  inefficient;  a  single  sending  node 
must  transmit  the  entire  message  separately  to  each  receiving  node  in  the  group,  as  in 
Figure  1.  This  causes  unicast  distribution  to  scale  poorly. 

Effective  group  communication  protocols  must  be  able  to  transmit  large  amounts 
of  data  with  a  relatively  small  amount  of  overhead.  For  example,  a  GPS  position  for  a 
location  of  importance  could  be  distributed  to  large  group  of  nodes  by  simply  setting  up  a 
unicast  link  to  each  node  without  regard  to  its  inefficiency  since  the  amount  of  data  is 
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small.  However,  a  video  stream  will  overwhelm  a  node  trying  to  unicast  the  stream  to 
even  a  small  group  because  the  sender  must  individually  encrypt  and  send  the  data  to 
each  receiving  node. 


Figure  1.  Unicast  distribution. 

Effective  group  communication  must  also  be  able  to  handle  large  groups.  Each 
additional  receiver  in  a  group  is  potentially  another  copy  of  the  message  that  must  be 
encrypted  and  sent.  Average  delay  to  send  a  message  grows  exponentially  as  the  amount 
of  data  approaches  network  capacity.  Thus,  while  it  is  feasible  to  unicast  distribute  small 
amounts  of  information  to  a  small  group,  but  problems  arise  and  are  magnified  with 
bigger  groups  and  larger  data  sizes. 

Another  factor  is  an  expectation  of  timeliness.  Data  sent  using  group 
communication  is  assumed  to  be  useful  when  received.  For  example,  the  usefulness  of 
threat  position  data  is  nil  if  the  threat  has  already  been  encountered.  For  group 
communication  to  be  effective  it  must  be  timely,  able  to  handle  large  amounts  of  data, 
and  able  to  scale  to  large  groups. 

2.2.2. 1  Multicast 

A  multicast  communication  protocol  is  addresses  some  of  the  scalability  issues  in 
group  communication.  Multicast  differs  from  broadcast  in  that  multicast  messages  are 
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intended  for  a  specified  group  of  nodes.  Ideally,  a  node  sends  a  multicast  message  once 
and  it  is  reliably  received  by  all  of  the  other  nodes  in  the  group  with  no  duplicate  packets 
being  transmitted. 


Figure  2.  Multicast  tree  distribution 

A  constant  load  send  method  in  group  communication  forms  a  tree  between  a 
sender  and  receivers,  as  in  Figure  2.  When  a  send  request  is  made,  the  receiver  of  the 
message  forwards  it  to  each  of  the  children  in  the  multicast  tree.  The  children  that 
receive  the  message  then  forward  the  message  to  their  children  which  continues  until  all 
nodes  receive  the  message. 

The  multicast  tree  has  two  major  drawbacks.  First,  if  a  node  fails,  all  child  nodes 
will  no  longer  receive  messages.  Second,  some  knowledge  of  the  topology  of  the  nodes 
must  be  known  to  establish  the  multicast  tree.  Otherwise,  the  result  may  be  more  of  a 
logical  overlay  than  it  is  a  multicast  tree  which  may  not  be  timely  or  efficient.  Finally, 
this  method  lacks  mechanisms  to  ensure  that  it  is  robust  or  scalable. 
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22.2.2  Mobile  Ad-hoc  Networks 


Networked  devices  are  expanding  beyond  the  static  world  of  traditional  wired 
networking.  Wireless  network  technology  has  become  widely  available,  and  it  allows 
devices  to  communicate  with  one  another  untethered  by  wires.  Furthermore,  devices  not 
only  stay  connected  to  each  other,  but  have  the  ability  to  move  and  discover  other 
devices.  These  dynamic  networks  are  known  as  mobile  ad-hoc  networks  (MANETs). 

Along  with  the  benefits  of  MANETs,  there  are  complications  that  must  be 
properly  addressed.  One  such  complication  is  the  reliability  of  the  wireless  connections. 
Data  transmitted  wirelessly  is  typically  sent  via  radio  frequencies  (RF).  Examples 
include  802. 1 1  and  Bluetooth.  RF  transmission  is  prone  to  many  types  of  failures  that  are 
not  as  common  in  wired  transmissions. 

Another  shortcoming  of  MANETs  is  the  mobility  of  the  nodes  in  the  network.  A 
node  that  was  previously  connected  to  a  group  of  nodes  might  encounter  an  obstacle  that 
prevents  communication.  The  break  in  communication  may  be  brief  or  extended  and 
may  cause  the  node  to  become  isolated.  Sometimes  the  node  may  still  be  connected  but 
via  different  links. 

The  unreliability  of  wireless  links  in  addition  to  problems  caused  by  the  roaming 
nodes  leads  to  a  high  probability  that  nodes  will  disconnect  from  the  network  and  connect 
at  a  later  time.  Roaming  nodes  may  also  join  and  leave  various  connected  components  of 
the  network  over  time  as  they  move.  The  rate  of  join  and  leave  events  in  the  network  is 
known  as  the  churn  rate.  A  network  with  a  high  chum  rate  has  difficulty  in  maintaining 
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reliability  because  in  attempting  to  achieve  reliability  in  an  unreliable  environment,  it 
increases  the  overhead  of  the  network. 

2.2.3  Secure  Group  Communication 

A  group  must  have  a  key  distributed  to  all  of  its  member  nodes  to  establish  secure 
communication.  To  satisfy  forward  and  backward  confidentiality,  the  group  key  must 
change  on  each  join  and  leave.  This  can  be  accomplished  on  a  join  by  using  the  previous 
group  key  to  distribute  the  new  group  key  and  using  secure  unicast  to  send  the  new  key  to 
the  joining  node.  Processing  a  leave  is  more  difficult  as  the  previous  key  cannot  be  used 
since  the  leaving  member  can  retain  the  previous  key  and  decrypt  further  communication 
using  the  key.  A  typical  solution  sends  the  new  group  key  to  each  remaining  node 
independently  using  multiple  unicast  secure  transmissions.  Mittra  describes  these  issues 
as  “1  affects  n”  failures  on  joins  and  both  “1  affects  n”  and  “1  does  not  equal  n”  failures 
on  leaves  [Mittra].  “1  affects  n”  is  a  failure  where  a  single  event  requires  an  action  to  be 
taken  for  all  other  members  in  a  group.  A  “1  does  not  equal  n”  failure  is  a  situation 
where  a  group  cannot  be  acted  on  as  a  group  and  each  member  must  be  considered 
separately. 

2.2.4  Required  Trust 

With  no  restrictions  on  topology,  any  node  in  a  group  has  the  ability  to  covertly 
establish  a  subgroup  not  authorized  by  the  group.  The  node  that  is  a  member  of  the  group 
can  distribute  all  decrypted  messages  it  receives  to  its  covert  subgroup.  Mobile  ad-hoc 
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networks  make  this  even  easier  since  a  covert  channel  can  be  created  simply  by  changing 
frequency. 

Due  to  this  inherent  vulnerability,  the  entire  membership  of  a  group  must  be 
trusted  to  maintain  the  membership  of  the  group  appropriately.  This  means  that  all  nodes 
must  be  infonned  of  the  group  membership  so  they  can  make  the  appropriate  choices  as 
to  send  or  not  send  a  message  or  key  to  another  node. 

2.3  Current  Research 

2.3.1  Iolus 

The  Iolus  protocol  [Mittra]  is  a  method  for  alleviating  “1  affects  n”  and  “1  does 
not  equal  n”  failures  by  breaking  multicast  groups  into  smaller  subgroups  and  organizing 
the  subgroups  into  a  hierarchical  distribution  tree.  Each  subgroup  has  a  group  security 
agent  (GSA)  that  maintains  keying  material  specific  to  that  subgroup.  The  subgroups  are 
joined  together  by  the  GSAs  that  are  members  of  a  subgroup  higher  in  the  tree  in  addition 
to  their  own  subgroup.  They  are  responsible  for  providing  communication  between  the 
two  groups  as  well  as  providing  communications  for  the  membership  of  the  subgroup 
they  are  serving.  Figure  3  shows  an  Iolus  group  configuration. 

To  bridge  a  subgroup,  message  encrypted  using  one  subgroup’s  key  is 
reencrypted  using  to  the  key  of  the  other  group.  To  avoid  decrypting  every  message  to 
plain  text  and  encrypting  the  plain  text  for  the  other  subgroup,  a  message  key  can  be  used 
to  encrypt  the  message  so  only  the  message  key  must  be  reencrypted.  The  GSA  is 
responsible  for  processing  all  joins  and  leaves  in  the  subgroup. 
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Figure  3.  Iolus-based  network. 

The  proposed  method  to  process  a  join  is  to  multicast  the  new  key  using  the 
current  group  key  to  the  current  membership.  The  GSA  must  then  establish  a  unicast 
secure  channel  to  the  joining  node  and  send  the  new  key  through  the  established  channel. 
This  channel  is  kept  open  for  the  duration  of  the  nodes  membership  in  the  subgroup. 
Thus,  the  GSA  must  keep  a  list  of  all  of  the  open  secure  channels  to  each  of  the  subgroup 
members. 

A  leave  is  more  expensive  as  the  current  key  cannot  be  used.  The  leave  process 
creates  a  new  key  and  sends  it  to  each  of  the  members  in  the  group  through  the  secure 
unicast  channel.  An  alternative  approach  encrypts  the  new  subgroup  key  using  the  key 
for  each  secure  channel  and  merges  the  result  into  one  large  message,  which  is  multicast 
to  the  subgroup. 
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Iolus  reduces  the  effect  of  failures  caused  by  joins  and  leaves,  but  relies  on  a 
single  intennediary  to  bridge  subgroups.  This  does  not  suit  mobile  ad-hoc  networks  well 
since  the  failure  of  a  single  node  or  an  intennittent  link  can  partition  the  network.  Iolus 
also  does  not  appear  to  take  advantage  of  overlapping  groups.  Each  group  must  be 
maintained  by  a  new  set  of  GSAs.  At  the  very  least,  GSAs  must  keep  separate  keying 
material  and  access  control  list  (ACL)  for  every  group. 

2.3.2  Cipher  Sequences 

Cipher  sequences  address  some  of  the  issues  of  Iolus  [Molva],  The  group  is 
divided  into  sub  groups  similar  to  Iolus.  However,  instead  of  being  connected  to  other 
subgroups  via  an  intermediary,  sub  groups  are  connected  to  the  root  node  through  a  series 
of  intennediate  nodes,  called  inner  nodes.  Each  of  the  inner  nodes  is  given  part  of  a 
cipher  sequence.  As  a  message  is  sent  through  the  inner  nodes,  each  node  applies  its 
piece  of  the  cipher  to  the  message.  Figure  4  shows  two  cipher  sequences  in  one  map. 


Figure  4.  Two  Cipher  Sequences 


A  benefit  of  this  approach  is  that  the  intermediaries  are  prevented  from  handling 
plain-text  data  in  the  manner  that  GSAs  must  do  in  Iolus.  This  is  important  because,  in 
contrast  to  Iolus,  inner  nodes  are  not  members  of  the  group.  The  inner  nodes’  only 
function  is  to  encode  the  cipher  sequences  represented  by  the  “f  ’  characters  in  Figure  4. 
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A  related  benefit  of  cipher  sequences  is  that  messages  are  encrypted  separately  for 
each  group.  Each  leaf  is  connected  to  the  root  through  a  unique  set  of  inner  nodes  so  the 
resulting  encryption  of  the  data  by  the  inner  nodes  is  unique.  This  isolates  the  effect  of 
any  compromised  node  to  a  single  leaf.  The  compromise  of  one  leaf  does  not 
compromise  the  remaining  leaves. 

2.3.3  Fast  Authenticated  Key  Establishment 

Huang  et  al.  proposes  a  method  for  nodes  to  efficiently  self-authenticate  and  share 
a  key  [Huang]  by  setting  up  each  node  offline  prior  to  deployment.  A  certificate  is 
generated  with  the  help  of  a  trusted  computation  server.  The  certificate  is  just  a  public 
key,  device  ID,  and  expiration  date  that  has  been  signed  by  the  certificate  authority. 
Elliptical  Curve  Cryptography  is  used  to  take  advantage  of  its  small  key  sizes  and  low 
communication  complexity.  The  protocol  is  designed  for  distributed  sensor  networks  and 
is  intended  to  lower  the  communication  and  computational  load  on  each  sensor  since 
sensor  nodes  have  limited  computational  ability  and  are  often  power  constrained.  The 
protocol  uses  certificates  when  the  nodes  are  online  so  nodes  can  quickly  authenticate 
and  negotiate  a  shared  symmetric  key. 

2.3.4  Antigone 

The  security  architecture  setup  by  McDaniel  [McDaniel]  gives  a  full  set  of 
security  mechanisms  that  can  be  chosen  and  combined  to  provide  the  policies  desired  for 
the  group.  There  are  mechanisms  for  joins,  leaves,  failure  detection,  authentication, 
application  messaging,  and  group  rekey/membership.  The  most  interesting  of  these  is  the 
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rekey/membership  mechanism.  The  mechanism  is  a  joint  message  that  has  the  new  key 
attached  with  the  membership.  This  is  shown  in  (a)  of  Figure  5.  The  joint  message  is 
encrypted  for  each  receiver  separately,  concatenated,  and  sent  out  as  one  large  rekey 
message  to  the  entire  group.  Figure  5  shows  a  full  rekey  message  for  Antigone  in  (b). 

a. 

| Joint  Message 
[i<EY 

b. 

|Encrypted  for  Node  1  |Encrypted  for  Node  2  |  ...  |Encrypted  for  Node  n 

[i<EY  [Membership  |KEY  [Membership  |  ...  |KEY  [Membership 

Figure  5.  a.  Antigone  Joint  Message  b.  Full  Antigone  Rekey  Message 
Rekeying  the  group  using  this  concatenated  joint  message  does  not  effectively 
scale  to  large  groups.  A  joint  message  grows  linearly  with  the  size  of  the  membership. 

In  addition,  the  number  of  joint  messages  that  are  concatenated  grows  linearly  with  the 
size  of  the  membership  because  a  joint  message  is  encrypted  and  concatenated  for  every 
member.  Thus,  the  size  of  the  rekey  message  grows  exponentially. 

2.3.5  Secure  Channel  Caching 

The  SecChan  layer  [Rodeh]  provides  secure  unicast  channels.  The  layer  keeps 
negotiated  symmetric  keys  associated  with  current  group  members  cached  to  avoid  the 
expense  of  reestablishing  the  secure  channel.  Channels  are  dropped  from  the  cache  if 
they  are  no  longer  associated  with  a  current  group  member  or  if  they  are  still  active  after 
a  set  expiration  period  to  prevent  cryptanalysis.  This  process  of  caching  symmetric 
secure  channels  avoids  the  cost  of  establishing  new  connections  at  the  expense  of  storage. 


[Membership 
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2.3.6  Gossip  using  Catalog  Messages 

A  gossip-based  network,  like  the  anti-entropy  phase  of  bimodal  multicast 
[Birman],  is  designed  for  intermittent  links.  Node  A  selects  another  node  at  random, 
Node  B,  and  sends  a  catalog  of  recently  received  messages  to  the  node,  as  in  Figure  6. 
Node  B  checks  to  ensure  it  has  received  all  messages  in  the  catalog,  and  requests  any 
messages  in  the  gossiped  catalog  that  it  has  not  already  received.  Node  A  then  sends  the 
requested  messages. 

Gossip  relies  on  the  random  spreading  of  messages  in  lieu  of  a  brute-force  method 
of  total  delivery.  Thus,  a  gossip  network  adapts  to  intennittent  links  keeping  live  nodes 
updated  and  attempting  to  keep  ill  nodes  updated  as  much  as  possible. 


Node  A 


Catalog1', 


Messages 


Node  B 


Request 


Check  Buffer 


Figure  6.  A  single  gossip 

Gossip  also  avoids  logical  single  points  of  failure.  The  random  selection  of  nodes 
does  not  limit  logical  groups  to  be  singly  connected  at  any  point  although  it  is  possible 
that  single  points  of  failure  remain,  at  the  physical  level. 


2.3.7  Gossip  as  a  Self-Sufficient  Protocol 

Gossip  [Birman]  is  a  part  of  a  larger  protocol.  Gossip  ensures  that  all  of  the  nodes 
in  a  group  have  the  same  messages.  Gossip  has  since  been  modified  to  be  the  entire 
message  distribution  scheme.  As  the  sole  message  distribution  method,  the  likelihood  of 
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a  node  not  having  all  messages  increases.  This  coupled  with  the  likelyhood  of  small 
messages  motivated  changing  gossip  to  send  the  actual  messages  in  place  of  the  catalog. 
This  eliminates  a  receiving  node  sending  a  request  for  messages  as  well  as  the  sending 
node  sending  a  response  which  cuts  the  load  of  new  messages  from  three  transmissions  to 
one. 

2.4  Summary 

This  chapter  discusses  current  research  in  group  communication  and  security 
protocols.  Message  keys,  secure  channel  caching,  joint  key/membership  messages,  and 
gossip  networks  from  this  chapter  are  used  in  the  following  chapters  to  introduce  a  novel 
approach  to  key  distribution  in  mobile  ad-hoc  networks. 
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III.  Methodology 


3.1  Problem  Definition 

MANETs  are  inherently  less  reliable  than  wired  networks.  The  ability  of  nodes  to 
achieve  the  connectivity  and  bandwidth  that  would  be  possible  if  no  link  failure  or 
interference  existed,  can  vary  greatly  in  a  group.  Mobility  alone  causes  significant 
connectivity  problems.  A  node  may  be  experiencing  interference  due  to  neighboring 
transmitters.  Moving  behind  an  obstruction  such  as  a  mountain  or  building,  could  cause  a 
drop  in  signal. 

Reliable  group  communication  protocols  often  expend  network  resources  to 
improve  reliability.  Probabilistic  multicast  protocols,  like  gossip,  expend  resources  to 
keep  healthy  nodes  updated.  Unhealthy  nodes  are  updated  when  possible,  but  these 
nodes  may  not  have  the  same  level  of  consistency  that  they  would  have  with  a  reliable 
multicast  protocol.  The  goal  of  a  gossip  protocol  is  to  increase  availability  by  ensuring 
that  “healthy”  nodes  receive  updates  with  high  probability.  “Unhealthy”  nodes 
communicate  as  well  as  they  are  able  and  do  not  cause  the  failure  of  “healthy”  nodes,  in 
probabilistic  multicast  systems  like  gossip. 

A  security  protocol  for  mobile  wireless  communication  can  be  built  on  similar 
principles  as  the  gossip  communication  protocol  to  increase  system  availability  at  the  cost 
of  inconsistency  in  unreliable  nodes  as  well  as  match  the  expected  losses,  faults,  and 
join/leave  events  in  wireless  environments.  With  gossip,  that  means  that  the  security 
protocol  must  expect  a  highly  dynamic  environment  and  should  not  be  greatly  affected  by 
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unhealthy  nodes.  The  offered  load  of  the  security  protocol  should  be  distributed  across 
the  network  to  increase  system  scalability. 

3.1.1  Goals  and  Hypothesis 

This  research  improves  the  scalability  of  a  secure  mobile  ad-hoc  network.  By 
using  concepts  from  catalog-based  gossip  networks  [Birman],  the  load  of  key  and 
message  dissemination  is  dispersed  among  the  members  of  the  group  thereby  reducing 
the  impact  of  intermittent  node  failure  on  the  scalability  of  the  network. 

3.2  Approach 

By  combining  established  ideas,  this  section  proposes  a  novel  key  distribution 
protocol  for  mobile  ad-hoc  networks.  The  general  idea  is  similar  to  the  catalogs  used  by 
initial  versions  of  gossip  with  the  actual  messages  acting  as  catalogs  of  the  required  keys. 
The  cached  unicast  security  channels  of  SecChan  [Rodeh]  reduce  computational  delays 
and  key  negotiation  delays  between  nodes.  The  combination  forms  a  powerful  protocol, 
which  maintains  network  security  with  relatively  low  overhead  and  has  high  resilience  to 
environments  prone  to  churn  like  mobile  networks. 

3.2.1  Secure  Unicast  Links 

It  would  be  possible  to  avoid  key  distribution  in  gossip  by  using  unicast 
encryption  schemes  for  each  gossip  message.  In  the  unicast  encryption  scheme,  a 
sending  node  encrypts  a  gossip  message  before  sending  the  message  to  the  receiver.  The 
receiver  then  decrypts  the  message  before  it  can  be  delivered  or  gossiped.  Establishment 
of  the  unicast  channel  must  also  be  included  in  this  process.  In  addition  to  the  inherent 
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delay  in  this  process,  there  is  no  way  to  account  for  the  membership  of  the  group  which 
must  be  attached  to  every  message  inside  of  a  gossip  message.  Consider  the  situation  in 
Figure  7. 


Figure  7.  A  single  message  being  gossiped  without  message  keys 


Node  A  is  sending  a  gossip  message  to  node  B.  Node  A  must  first  encrypt  the 
message  with  the  key  for  node  B.  While  node  A  is  encrypting  the  message,  node  B 
decides  to  gossip  to  node  C.  The  gossip  to  node  C  does  not  contain  the  message  from 
node  A  because  it  has  not  been  sent  yet.  In  fact,  the  message  will  not  be  included  in 
gossip  messages  from  node  B  until  after  the  message  has  been  delivered  to  node  B.  At 
some  point  after  the  message  has  been  delivered  to  node  B,  node  B  again  decides  to 
gossip  to  node  C.  The  message  is  now  included  in  the  gossip  message.  As  before,  node 
B  must  encrypt  the  message  with  the  key  for  node  C  before  the  message  can  be  sent  and 
must  be  decrypted  before  the  message  can  be  delivered  or  gossiped. 

3.2.2  Minimize  Encryption 

Similar  to  Iolus,  the  originator  of  a  message  is  like  the  group  security  controller 
(GSC)  for  the  group  and  other  nodes  in  the  group  are  like  group  security  intennediaries 
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1  Send() 

2  Generate  message  key 

3  Encrypt  message  with  key 

4  Attach  message  key  ID  to  encrypted  message 

5  Add  message  to  buffer 
5  End 


Figure  8.  Send  procedure 


1 

Receive  ( ) 

2 

Receive  encrypted 

message  buffer 

3 

For  (messages  in  new  buffer) 

4 

If  (message  not 

in  current 

buffer) 

5 

Add  encrypted 

message  to 

current  buffer 

6 

End- if 

7 

If  (message  key 

not  in  key 
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8 

Send  message 

key  request 

9 

End- if 

10 

End-f or 

11  End 

Figure  9.  Receive  procedure 

(GSI).  The  send  procedure  using  pseudo-code  is  shown  in  Figure  8.  The  originator 
creates  a  symmetric  key  to  be  used  as  the  message  key,  Km.  The  originator  encrypts  the 
message  (M)  using  the  Km  giving  e(M,Km).  A  key  identifier  is  also  added  to  the 
encrypted  message  which  consists  of  the  address  of  the  originator  and  a  unique  identifier 
assigned  by  the  originator.  There  are  no  duplicate  key  identifiers  are  due  to  the  inclusion 
of  the  originator’s  address.  Now  e(M,Km)  +  KeylD  can  be  included  in  gossip  messages 


1 

Deliver  ( ) 

2 

Decrypt 

Message  with  Message  Key 

3 

Deliver 

Message 

4 

End 

Figure  10.  Message  Delivery 


without  needing  further  encryption.  The  receive  procedure  in  Figure  9  does  not  contain 
message  decryption  as  the  decryption  task  is  delayed  until  message  delivery  as  shown  in 
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Figure  10.  Encrypting  the  message  with  a  message  key  prevents  the  message  from  being 
reencrypted  by  nodes  in  the  group,  reducing  the  amount  of  data  that  must  be  encrypted 
and  decrypted  to  only  the  message  key. 


1  Key  Received () 

2  Decrypt  Key 

3  If  (Key  not  in  key  cache) 

4  Add  Key  to  message  key  cache 

5  End-if 

6  End 


Figure  11.  Receiving  a  message  key 

A  node  receiving  a  gossip  message  must  determine  by  examining  the  messages 
contained  in  the  gossip  which  keys  it  needs.  The  node  then  sends  a  request  for  the 
needed  keys  to  the  originator  of  the  gossip  message.  The  sender  replies  with  the 
requested  keys  over  a  secure  channel  as  shown  in  Figure  12  to  be  processed  by  the 
requester  as  in  Figure  1 1 . 


1  Key  Request  Received ( ) 

2  If  (message  key  not  in  key  buffer) 

3  Ignore  key  request 

4  Else 

5  If  (request  node  is  not  in  key  ACL) 

6  Send  key  request  failure 

7  Else 

8  If  (secure  channel  is  not  in  cache) 

9  Establish  secure  channel 

10  End-if 

11  Send  key  via  secure  channel 

12  End-if 

13  End-if 

14  End 


Figure  12.  Receiving  a  key  request 

In  Figure  13,  a  gossip  message  sent  immediately  following  node  A  gossiping  to 
node  B.  Node  B  receives  the  message  and  adds  the  message  to  the  buffer  of  messages  to 
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be  gossiped.  This  is  possible  because  the  message  is  still  encrypted  with  the  message  key 
and  must  be  encrypted  such  to  be  gossiped.  Node  B  must  then  request  the  message  key 
from  node  A.  Differing  from  the  previous  example,  when  node  B  decides  to  gossip  to 
node  C,  the  message  is  included  in  the  gossip  even  though  node  B  has  not  yet  received 
the  message  key  or  delivered  the  message.  Thus,  node  C  receives  the  message  much 
quicker.  Node  C  then  requests  the  message  key  from  node  B. 


Node  A 


Figure  13.  A  single  message  being  gossiped  with  message  keys 


3.2.3  Membership 

As  seen  in  Figure  12,  a  node  that  forwards  a  message  key  must  know  who  is 
allowed  to  receive  the  key.  Therefore,  to  establish  the  membership  of  the  group  to  which 
the  message  is  sent,  the  originator  attaches  the  access  control  list  (ACL)  to  the  message 
key,  similar  to  Antigone’s  joint  key  [McDaniel].  Thus,  each  key  has  the  group 
membership  attached.  This  has  increases  the  amount  of  data  that  must  be  reencrypted  at 
each  node  to  the  size  of  Km  and  that  of  the  ACL. 

A  node  might  not  have  received  a  key  that  has  been  requested  by  another  node. 
Consider  a  node,  NB,  that  receives  a  gossip  message  and  send  a  request  for  a  message  key 
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that  it  does  not  have  to  Na.  Nb  may  gossip  the  same  message  to  another  node,  Nc,  which 
also  does  not  have  the  required  message  key.  Nc  could  send  a  key  request  to  NB  before 
Nb  receives  the  message  key  it  requested  before.  In  this  case,  NB  drops  the  key  request 
and  does  not  respond  to  it.  Nc  interprets  this  as  a  lost  message  and  will  try  the  request 
again.  Recall  that  to  overcome  broken  links,  Nc  will  probabilistically  choose  to  request 
the  message  key  from  either  NB  or  the  originator  of  the  message  key. 

3.2.4  Caching  Secure  Unicast  Channels 

The  gossip  protocol  relies  heavily  on  secure  unicast  channels  between  nodes. 
Uunicast  channels  have  the  benefit  of  being  usable  across  overlapping  groups  so  two 
nodes  belonging  to  the  same  groups  can  use  the  established  secure  unicast  channel  for 
communication  requiring  such  a  channel  in  all  groups  they  share  membership.  For 
example,  nodes  NA  and  NB  are  members  of  both  group  Gi  and  group  G2.  They  only  need 
to  establish  a  channel  between  them  one  time  to  allow  sending  of  secure  data  for  both 
groups  Gi  and  G2. 

To  further  optimize  the  secure  unicast  channels,  public  key  encryption  can  be 
avoided  by  using  a  method  similar  to  Rodeh’s  SecChan  layer  [Rodeh].  A  node  pair 
negotiates  a  symmetric  link  key  (Kl)  for  sending  secure  data.  Each  node  keeps  a  cache 
of  negotiated  symmetric  keys  so  the  pair  does  not  need  to  renegotiate  a  new  symmetric 
key  for  each  use  of  a  secure  channel.  This  reduces  the  processing  involved  in 
reencrypting  message  keys  because  the  decryption  and  encryption  both  are  done  using 
less  expensive  symmetric  processes. 
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The  separate  link  encryption  by  each  node  pair  takes  advantage  of  the  physical 
attributes  of  different  links.  A  node  might  handle  the  unicast  link  encryption  differently 
over  laser  link  than  it  would  over  an  RF  link.  Since  a  sender  transmits  the  message  key 
over  a  laser  link,  encryption  may  be  deemed  unnecessary  as  the  probability  of 
interception  is  extremely  low.  This  eliminates  the  processing  expense  of  encrypting  the 
message  key  on  the  sender  and  decrypting  the  message  key  on  the  receiver. 

3.3  System  Boundaries 

The  system  under  test  consists  of  the  simulated  environment  in  which  the 
scenarios  are  run.  Included  are  the  security  and  transmission  protocols  as  well  as 
simulated  mechanisms  to  send,  transport  and  receive  messages.  The  distribution 
schemes,  Iolus  and  gossip  with  on-demand  key  distribution,  make  up  the  component 
under  test. 

3.4  System  Services 

The  system  provides  secure  delivery  of  data  to  a  group  of  nodes.  The  system 
provides  key  distribution  to  accomplish  this  end.  The  system  is  fully  successful  when  a 
message  is  delivered  to  all  members  of  a  group.  For  MANETs,  this  may  be  more 
appropriately  restricted  to  delivery  to  all  the  healthy  nodes  of  a  network.  The  system  fails 
when  nodes  cannot  deliver  a  message.  This  can  be  due  to  either  not  receiving  a  message 
intended  for  the  node  or  not  receiving  the  key  needed  to  decrypt  the  message  so  that  it 
can  be  delivered.  The  system  also  fails  when  a  message  is  delivered  by  a  node  for  which 
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the  message  was  not  intended.  This  requires  that  nodes  receive  both  the  encrypted 
message  and  the  key  or  that  a  node  sends  a  plaintext  message. 

3.5  Workload 

The  offered  load  is  the  number  of  sent  messages  per  second,  20  messages  per 
second.  Messages  to  be  delivered  to  all  nodes  in  the  group  must  be  sent  securely 

3.6  Performance  Metrics 

•  Percentage  of  Messages  Delivered  -  This  is  the  percentage  of  group  members  that 
received  a  particular  message.  When  comparing  a  probabilistic  multicast  protocol 
to  a  reliable  protocol,  the  impact  on  message  delivery  is  critical. 

•  End-to-end  delay  -  This  is  the  time  from  when  the  message  is  submitted  to  the 
system  to  the  time  that  a  message  and  its  key  is  received  at  a  node.  This  is  the 
amount  of  time  that  a  message  is  in  the  system. 

•  Load  -  This  is  a  measure  of  the  load  on  nodes  in  the  network  in  terms  of  the 
number  of  messages  (or  bytes)  per  unit  of  time.  Of  particular  interest  is  the 
maximum  load  and  load  standard  deviation  which  indicates  how  well  the  load  of 
the  network  is  distributed  to  the  group. 

3.7  System  Parameters 

•  Number  of  group  s/senders  -  The  number  of  groups  affects  how  nodes  interact 
with  each  other  and  the  amount  of  data  entering  the  network. 

•  Topology  -  The  network  topology  determines  the  amount  of  work  required  to 
transmit  data.  The  topology  for  all  scenarios  is  set  up  as  shown,  or  is  similar  to 
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Figure  14.  For  101  node  scenarios,  there  are  20  nodes  at  each  level.  For  2  group 
scenarios,  the  topology  still  has  20  nodes  at  each  level,  but  also  has  an  additional 
sender.  Each  group  contains  12  nodes  from  each  level  with  the  middle  4  nodes  at 
each  level  in  both  groups. 


Figure  14.  51  node  mesh  topology 


3.8  Workload  Parameters 

•  Node  Failure  Rate  -  The  number  of  failures  per  unit  of  time  detennines  how 
much  work  the  network  must  do  to  compensate 

•  Group  Size  -  The  group  size  affects  how  widely  a  message  must  be  distributed. 

•  Message  Rate  -  The  rate  that  messages  enter  the  system  directly  affects  the  load 
placed  on  the  system. 

3.9  Factors 

Each  of  the  scenarios  in  Table  1  is  run  as  a  simulated  Iolus  distribution  system 
and  then  again  with  a  gossip  message  distribution  system  with  an  on-demand  key 
distribution.  The  key  distribution  scheme  replicates  a  workload  as  it  might  be 


27 


experienced  in  a  mobile  ad-hoc  network,  and  the  factors  were  chosen  in  consideration  of 
such  an  environment.  The  node  failure  rate  is  likely  the  most  important  factor  due  to  its 
prevalence  in  MANETs.  Other  important  factors  include  group  size  and  the  number  of 
groups.  Table  1  lists  the  settings  for  each  of  the  factors. 


Table  1.  Factors  Settings 


Scenario 

Messages  per 
second 

Nodes 

Groups 

Group  Size 

Node  Failure  Rate 

1 

20 

51 

1 

51 

0% 

2 

20 

51 

1 

51 

3% 

3 

20 

51 

1 

51 

10% 

4 

20 

101 

1 

101 

0% 

5 

20 

101 

1 

101 

3% 

6 

20 

101 

1 

101 

10% 

7 

20 

102 

2 

61 

0% 

8 

20 

102 

2 

61 

3% 

9 

20 

102 

2 

61 

10% 

3.10  Evaluation  Technique 

The  key  distribution  protocol  described  for  gossip  and  Iolus  use  network 
simulator  2  (NS2).  NS2  is  a  network  simulation  program  written  in  C++  and  TCL 
[BreslauOO].  NS2  simulates  communication  between  nodes  for  a  user-defined  topology 
which  are  set  up  using  TCL  scripts. 

3.11  Validation 

There  currently  are  no  published  results  for  the  Iolus  protocol  to  validate  against. 
However,  distinct  load  characteristics  are  anticipated  for  Iolus  results.  The  load  is 
expected  to  focus  on  a  small  set  of  nodes,  the  GSAs.  This  will  give  high  maximum  load 
values  relative  to  the  mean  load  per  node. 
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3.12  Experimental  Design 

The  experiments  include  the  2  message  distribution  schemes  (Iolus  and  gossip 
with  on-demand  key  distribution),  3  node  fault  rates,  3  group  configurations,  and  10 
trials.  This  provides  18  experiments  of  10  trials  each  giving  a  total  of  180  experiments. 

3.13  Summary 

This  chapter  describes  an  experiment  to  analyze  the  effectiveness  and  scalability 
of  message  distribution  schemes  for  mobile  ad-hoc  networks.  Measuring  the  impact  on 
end-to-end  delay  and  the  load  of  varying  node  failure  rates  allows  an  assessment  of  the 
scalability  of  the  distribution  scheme.  Measuring  the  percent  of  nodes  receiving  the 
messages  determines  at  what  cost  that  scalability  comes. 
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IV.  Results  and  Analysis 


4.1  Chapter  Overview 

This  chapter  compares  an  Iolus-based  distribution  to  a  gossip-based  distribution 
with  on-demand  key  distribution  using  the  scheme  from  Chapter  3.  The  end-to-end 
delay  and  load  on  the  network  are  used  to  analyze  the  scalability  of  each  system  and  the 
percentage  of  messages  received  is  used  to  determine  the  cost  of  using  an  unreliable 
protocol.  All  scenarios  were  run  for  300  seconds  with  a  120  second  fault  period  starting 
at  123.5  seconds.  All  charts  truncate  the  first  100  seconds  and  last  40  seconds. 

Additional  charts  not  mentioned  in  Chapter  4  can  be  found  in  Appendix  A. 

4.2  End-to-end  Delay 

The  first  metric  considered  is  the  end-to-end  (ETE)  delay  of  each  system.  Each 
message  was  timed  from  entering  the  system,  prior  to  any  encryption,  to  being  delivered 
at  each  node  which  means  the  node  must  have  both  the  encrypted  message  and  the 
message  key. 

Figure  15  shows  the  ETE  delay  for  the  5 1  node  gossip  scenarios.  Each  of  the 
three  scenarios  has  a  different  node  failure  rate.  From  the  chart,  average  ETE  delay 
increases  as  the  node  failure  rate  increases.  With  a  node  failure  rate  of  3%,  the  average 
ETE  delay  increases  about  5%  on  average.  Increasing  the  node  failure  rate  to  10%  results 
in  an  increase  of  about  15%  in  the  average  ETE  delay. 

The  Iolus  baseline  (0%  node  fault  rate)  scenario  has  a  much  lower  ETE  delay  than 
the  gossip  baseline.  However,  Iolus  is  more  sensitive  to  faults.  This  is  also  true  for  the 
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101  node  scenarios  shown  in  Figure  16.  The  increased  workload  due  to  the  number  of 
nodes  causes  the  average  ETE  delay  for  Iolus  with  the  10%  node  fault  rate  to  exceed  that 
of  the  gossip  equivalent.  The  2  group  scenarios,  in  Figure  17,  see  a  better  ETE  delay  for 
Iolus  likely  due  to  the  increased  number  of  senders.  However,  the  delay  has  a  similar 
sensitivity  to  the  node  fault  rate. 

Table  2  gives  a  summary  of  the  percent  increase  in  ETE  delay  for  each  of  the 
runs.  This  is  the  percent  increase  in  the  average  delay  over  the  120  second  node  failure 
time  compared  to  the  0%  node  failure  rate  scenario.  The  3%  node  failure  rate  gives 
between  a  59%  to  83%  increase  in  Iolus  ETE  delay,  and  the  10%  node  failure  rate  causes 
a  nearly  400%  increase  in  the  101  node  scenario.  The  gossip  scenarios  see  a  relatively 
constant  5%  increase  for  a  3%  node  fault  rate  and  about  15%  increase  in  ETE  delay  for 
the  10%  node  fault  rate  scenarios. 


51  Node  Scenarios 
Average  End-to-End  Delay 


—  Gossip  0% 

—  Gossip  3% 

—  Gossip  1 0% 

—  Iolus  0% 

—  Iolus  3% 

—  Iolus  10% 


Figure  15.  Average  End-to-End  Delay  for  51  node  scenarios 
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End-to-End  Delay  (s) 


101  Node  Scenarios 
Average  End-to-End  Delay 


Scenario  Time  (s) 


—  Gossip  0% 

—  Gossip  3% 
Gossip  10% 

—  lolus  0% 

—  lolus  3% 

—  lolus  10% 


Figure  16.  Average  End-to-End  Delay  for  101  node  scenarios 


2  Group  Scenarios 
Average  End-to-End  Delay 


- Gossip  0% 

—  Gossip  3% 

—  Gossip  1 0% 

—  lolus  0% 

—  lolus  3% 

—  lolus  10% 


Scenario  Time  (s) 


Figure  17.  Average  End-to-End  Delay  for  2  group  scenarios 
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Table  2.  Node  Failure  Rate  and  End-to-End  percent  increase  for  all  scenarios 


Group 

Node  Fault  Rate 

Iolus 

Gossip 

51  Nodes 

3% 

73% 

5% 

10% 

318% 

14% 

101  Nodes 

3% 

83% 

5% 

10% 

390% 

14% 

2  Group 

3% 

59% 

4% 

10% 

254% 

12% 

A  similar  trend  appears  in  the  standard  deviation  for  the  end-to-end  delay  in  the 
51  node  scenarios  as  shown  in  Figure  18.  The  standard  deviation  slightly  decreases  for 
gossip  likely  due  to  the  elimination  of  outliers  by  message  expiration.  The  standard 
deviation  for  Iolus  continues  to  increase  as  the  node  failure  rate  increases.  Charts  for  the 
ETE  delay  standard  deviation  for  101  node  scenarios  and  2  group  scenarios  display 
similar  characteristics  and  can  be  found  in  Appendix  A. 

Figure  19  shows  the  maximum  ETE  delay  for  the  5 1  node  scenarios.  For  the  10% 
node  failure  rate,  the  Iolus  maximum  ETE  distribution  is  as  high  as  38  seconds.  The 
maximum  ETE  delay  for  gossip  stays  at  about  2  seconds.  101  node  scenarios  and  2  group 
scenarios  also  exhibit  this  high  maximum  ETE  delay  shown  in  Figures  20  and  21, 
respectively.  The  maximum  delay  for  Iolus  peaks  as  high  as  30  seconds  in  the  101  node 
scenarios  and  25  seconds  for  the  2  group  scenarios.  Gossip  maximum  end-to-end  delay 
remains  at  2  seconds  for  both  sets  of  scenarios 
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End-to-End  Delay  Standard  Deviation  (s) 


51  Node  Scenarios 
Delay  Standard  Deviation 


Figure  18.  End-to-End  delay  standard  deviation 


51  Node  Scenarios 
Maximum  End-to-End  Delay 


Figure  19.  51  Node  Scenarios  Maximum  End-to-End  delay 


Gossip  0% 
Gossip  3% 
Gossip  10% 
lolus  0% 
lolus  3% 
lolus  10% 


Gossip  0% 
Gossip  3% 
Gossip  1 0% 
lolus  0% 
lolus  3% 
lolus  10% 
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End-to-End  Delay  (s)  End-to-End  Delay  (s) 


101  Node  Scenarios 
Maximum  End-to-End  Delay 


Figure  20.  101  Node  Scenarios  Maximum  End-to-End  delay 


2  Group  Scenarios 
Maximum  End-to-End  Delay 


Figure  21.2  Group  Scenarios  Maximum  End-to-End  delay 


Gossip  0% 
Gossip  3% 
Gossip  10% 
lolus  0% 
lolus  3% 
lolus  10% 


Gossip  0% 
Gossip  3% 
Gossip  10% 
lolus  0% 
lolus  3% 
lolus  10% 
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Figure  22  compares  the  average  time  to  distribute  the  message  key  for  each  of  the 
5 1  node  scenarios.  Gossip  has  very  minimal  increase  in  distribution  times.  The  key 
distribution  time  increases  from  about  150ms  for  0%  node  fault  rate  to  200ms  for  10% 
node  fault  rate.  Iolus  has  a  lower  key  distribution  time  averaging  100ms  for  the  0%  node 
fault  rate  case.  The  Iolus  key  distribution  delay  is  on  par  with  gossip  at  the  3%  rate. 
However,  Iolus  key  distribution  is  greatly  affected  by  the  10%  node  fault  rate  reaching  as 
high  as  700ms.  Charts  for  the  remaining  scenarios  can  be  found  in  Appendix  A.  Figures 
23  and  24  summarize  the  group  types  giving  the  average  and  maximum  key  distribution 
delays  for  the  10%  node  fault  rate  scenario  for  each  group  type.  The  gossip  key 
distribution  scheme  consistently  achieves  lower  key  distribution  delays  than  the  Iolus- 
based  scheme  under  higher  node  fault  rates. 


51  Node  Scenarios 

Average  Delay  to  Receiving  Message  Key 


—  Gossip  0% 
- Gossip  3% 

—  Gossip  10% 

—  Iolus  0% 

- Iolus  3% 

- Iolus  10% 


Figure  22.  Average  key  distribution  End-to-End  delay 
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Key  Delay  (s)  £  Key  Delay  (s) 


10%  Node  Fault  Rate  Scenarios 
Average  Delay  to  Receiving  Message  Key 


—  Gossip  51  Node 

—  Gossip  101  Node 
Gossip  2  Group 

—  lolus  51  Node 

—  lolus  101  Node 

—  lolus  2  Group 


23.  Average  message  key  distribution  End-to-End  delay  for  10%  node  fault 


10%  Node  Fault  Scenarios 
Maximum  Delay  to  Receiving  Message  Key 


—  Gossip  51  Node 

—  Gossip  101  Node 
Gossip  2  Group 

—  lolus  51  Node 

—  lolus  101  Node 

—  lolus  2  Group 


Figure  24.  Maximum  message  key  End-to-End  delays  for  10%  node  fault 


37 


4.3  Network  Load 


The  network  load  shows  how  the  work  is  distributed  to  all  of  the  nodes  on  the 
network.  Figures  25  and  26  present  the  average  and  maximum  message  send  rate  for 
each  of  the  5 1  node  scenarios.  The  average  for  each  of  the  gossip  was  a  constant  six 
messages  due  to  a  constant,  non-adaptive  gossip  rate.  In  Figure  26,  we  see  that  the 
message  send  rate  extends  as  high  as  500  messages  per  second  and  even  the  base  is 
greater  than  the  constant  gossip  rate. 

Figures  27  and  28  show  the  average  and  maximum  send  rates  in  bytes  sent  per 
node.  The  average  send  rate  in  bytes  in  the  gossip  scenarios  range  between  90,000  and 
100,000  bytes/s.  This  is  much  higher  than  the  average  Iolus  send  rate.  However,  as  with 
the  maximum  message  send  rate,  the  maximum  send  rate  in  bytes  for  Iolus  varies  greatly 
and  is  frequently  higher  than  the  send  rates  of  gossip.  At  one  point,  the  maximum  sent 
bytes  of  Iolus  is  about  double  the  maximum  sent  bytes  of  gossip. 

While  the  average  case  for  Iolus  appears  to  give  a  lesser  load  on  the  network, 
there  are  a  handful  of  nodes  that  are  doing  all  of  the  work.  This  can  be  seen  in  the 
comparison  of  the  average  send  rate  in  Figure  25  to  the  maximum  send  rate  in  Figure  28. 
Gossip  generally  distributes  more  data  than  Iolus,  but  this  load  is  spread  amongst  all  of 
the  nodes  in  the  network.  In  addition,  gossip  uses  a  smaller  amount  of  larger  messages  to 
distribute  data  due  to  the  use  of  message  buffers. 

Also  of  interest  in  Figures  24-30,  the  gossip  scenarios  exhibit  little  to  no  change 
in  send  rate  when  the  node  fault  rate  is  increased.  The  increased  node  failure  does  not 
increase  overhead  load  on  the  network. 
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Send  Rate  (packets/s)  Send  Rate  (packets/s) 


51  Node  Scenarios 
Average  Send  Rate 


—  Gossip  0% 

—  Gossip  3% 
Gossip  10% 
lolus  0% 
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—  lolus  10% 
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Figure  25.  Average  message  send  rate 


51  Node  Scenarios 
Maximum  Send  Rate 
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Figure  26.  Maximum  message  send  rate 
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Send  Rate  (bytes/s)  Send  Rate  (bytes/s) 


51  Node  Scenarios 
Average  Send  Rate 
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Figure  27.  Average  send  rate  per  node 


51  Node  Scenarios 
Maximum  Send  Rate 
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Figure  28.  Maximum  send  rate  per  node  in  5 1  node  scenarios 
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Send  Rate  (bytes/s) 


101  Node  Scenarios 
Maximum  Send  Rate 
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Figure  29.  Maximum  send  rate  per  node  in  101  node  scenarios 


2  Group  Scenarios 
Maximum  Send  Rate 
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Figure  30.  Maximum  send  rate  per  node  in  2  group  scenarios 
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4.4  Percent  of  Messages  Delivered 

The  cost  of  unreliable  communication  is  analyzed  by  looking  at  the  percent  of 
messages  that  were  not  delivered.  Figures  3 1,  32,  and  33  show  this  for  each  of  the  gossip 
scenarios.  This  is  not  given  for  Iolus  as  all  communication  is  reliable  (i.e.,  re-sent  until 
acknowledged).  The  0%  node  fault  rate  for  gossip  gives  average  reliability  above  99% 
for  all  group  types.  A  node  fault  rate  of  3%  lowers  the  reliability  to  98-97%.  The  10% 
node  fault  rate  case  goes  as  low  as  91%.  This  rate  is  influenced  by  the  message 
expiration  time  in  gossip.  If  desired,  higher  reliability  could  be  achieved  by  lengthening 
this  expiration  time. 


51  Node  Scenarios 
Gossip  Mean  Percent  Received 


Figure  31.  Percentage  of  gossip  messages  received  in  51  node  scenarios 
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Mean  Percent  Received  (%  of  nodes)  Mean  Percent  Received  (%  of  nodes) 


101  Node  Scenarios 
Gossip  Mean  Percent  Received 


Figure  32.  Percentage  of  gossip  messages  received  in  101  node  scenarios 


2  Group  Scenarios 
Gossip  Mean  Percent  Received 


Figure  33.  Percentage  of  gossip  messages  received  in  2  group  scenarios 
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4.5  Summary 

This  chapter  analyzes  the  results  from  the  experiment  setup  in  Chapter  3.  The 
gossip  system  has  a  reliability  of  over  90%  for  all  scenarios.  This  compares  to  the  Iolus 
system  that  was  designed  to  give  100%  reliability. 

While  Iolus  performs  better  in  the  stable  scenarios,  gossip  outperforms  Iolus  in 
for  increasing  node  failure  rates.  Iolus  experienced  ETE  delays  over  35  seconds  for  a 
10%  node  failure  rate  for  both  key  and  message  distribution.  Gossip  handled  node  failure 
rate  consistently  across  group  types.  The  10%  node  failure  rate  gave  only  a  15%  increase 
in  average  ETE  delays.  Under  the  same  failure  rate,  Iolus  displayed  a  250%-400% 
increase  in  ETE  delay. 

The  Iolus  system  has  a  lower  average  network  load,  but  appeared  to  have  hotspots 
where  the  load  was  drastically  higher  than  the  rest  of  the  group.  Gossip  had  a  higher 
average  load,  but  had  a  much  lower  difference  between  average  and  maximum  load  per 
node. 
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V.  Conclusion 


5.1  Chapter  Introduction 

The  typical  working  environment  of  MANETs  often  consists  of  frequent  node 
outages,  unreliable  links,  and  open  communication.  Transmitting  secure  data  in  this 
environment  calls  for  a  scalable,  low-overhead  key  distribution  scheme  that  delivers 
messages  quickly  to  healthy  nodes  while  coping  with  unhealthy  nodes.  Gossip  provides  a 
high-probability  of  delivery  basis  to  base  such  a  scheme  upon.  This  research  proposes 
and  analyzes  on-demand  key  distribution  based  on  gossip. 

5.2  Conclusion 

Gossip  with  on-demand  key  distribution  appears  to  scale  well  in  unreliable 
networks.  Gossip,  under  a  10%  node  failure  rate,  gave  only  a  15%  percent  increase  in 
average  ETE  delay  compared  to  Iolus  resulting  in  250%-400%  percent  increase  in  ETE 
delay.  Message  keys,  secure  channel  caching,  and  joint  keying  distribute  the  key  and 
message  distribution  workload  while  incurring  little  additional  overhead.  Gossip  with 
on-demand  key  distribution  distributes  messages  effectively  in  accord  with  the  node’s 
ability  to  receive. 

The  on-demand  key  distribution  scheme  and  gossip  tolerates  an  increasing  node 
fault  rate,  a  characteristic  typical  to  MANETs,  while  distributing  the  load  of  message  and 
key  distribution  to  all  of  the  nodes  in  the  group.  The  maximum  send  rate  for  gossip 
stayed  at  140  kB/s,  while  Iolus  peaked  at  300  kB/s. 
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5.3  Recommendations  for  Future  Research 


Wireless  networks  drive  the  need  for  further  development  of  secure  transmission 
protocols  which  are  both  scalable  and  fault-tolerant.  This  research  considered  a  mesh 
topology.  Further  research  could  analyze  the  performance  of  various  topologies. 

A  variety  of  established  unicast  security  protocols  could  be  incorporated  into  the 
key  distribution  scheme.  This  could  be  expanded  to  hybrid  networks  using  a  variety  of 
protocols  for  each  link  in  the  system. 

The  experiments  in  this  research  can  also  be  expanded  to  look  at  the  effect  of 
group  membership  changes.  A  rekey  is  forced  by  a  join  or  a  leave.  In  either  case,  the 
rekey  is  done  by  the  sender  changing  the  message  key  and  updating  the  key  ID  attached 
to  new  messages.  In  some  cases,  this  rekey  may  not  be  necessary.  For  example,  it  may 
be  inconsequential  whether  a  node  receives  the  remaining  part  of  a  message  when  they 
have  already  received  the  first  part.  This  can  be  left  to  the  application. 
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